The present disclosure generally relates to artificial reality systems, and specifically to distributed artificial reality systems with hand tracking devices.
Augmented reality (AR) systems and environments allow a user to directly or indirectly view a real world environment augmented by generated sensory input, which may be super-imposed on the real world environment. Sensory input can be any form of media, such as sound, video, graphics, etc. Typically, an AR system includes a headset that provides visual and audio information to the user. Because AR systems allow for users to continue to engage with their real world environments in addition to a generated one, users may have less tolerance for large AR headset devices, as opposed to a virtual reality (VR) system in which the user is typically immersed in a fully generated environment. Additionally, smaller form factors facilitate all-day or longer use of artificial reality systems by reducing the friction between a user's experience of his or her environment and the artificial reality system itself.
However, the reduced form factor of AR systems produces challenges for providing sufficient power and computation, and limits the capacity for adding additional features that may enhance the user's AR experience and facilitate the interaction of the AR system with the environment. Furthermore, hand gestures are an important way in which users interact with and indicate objects within their environment, but these are not captured by a headset device. Because of its limited field of view, a headset device may be unable to see a user's hands or capture the user's gestures in response to the simulated environment presented by the headset. Furthermore, conventional hand tracking systems focus primarily on simply capturing hand movements, rather than determining what a hand motion means in the context of the user's environment and other signals detected by the distributed system. More generally, distributed systems often fail to fully integrate different devices and leverage the ability of a distributed AR system to combine information captured by each of the devices in the system.